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BEST AVAILABLE IMAGES 



Defective images within this document are accurate representations of 
the original documents submitted by the applicant. 

Defects in the images may include (but are not limited to): 



BLACK BORDERS 

TEXT CUT OFF AT TOP, BOTTOM OR SIDES 
FADED TEXT 
ILLEGIBLE TEXT 
SKEWED/SLANTED IMAGES 
COLORED PHOTOS 

BLACK OR VERY BLACK AND WHITE DARK PHOTOS 
GRAY SCALE DOCUMENTS 



IMAGES ARE BEST AVAILABLE COPY. 



As rescanning documents will not correct images, 
please do not report the images to the 
Image Problem Mailbox. 
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Face Detection Workflow (Detection Module) 



.3110 



Wait for Image from Main, 
Workflow (Idle State) 



312 




7 3122 



Yej— — ^ |scan Image with Face Feature Pre-Filter j 

X 



Locate Face Pixel Groupings ^ 



[Locate & Mark Face Candidate Regions J 



Cluster Groupings to form 
Complete Face Candidate Regions 



\3124 



-3130 



| Add to Auto-Recognition List j 

3 1 50^" ~* ^ 



-3140 




Pass Auto-Recognition List to 
Workflow Module and return to Idle 



3160 



Cluster Remaining Face Pixel Groupings 
to Form Partial Face Candidates 



3162- 



-3170 



[ Add to Manual-Recognition List 




N 



,3192 



Merge to a single Training List J 
[pass Training List to Workflow Module 



Pass Manual & Auto-Recognition Lists 
to Workflow Module and return to Idle 



^3194 



^3200 



Fig 3 
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Core System Workflow (Workflow Module I) 



,4110 



Wait for Next Image 
(Main Workflow Idle) 



I * 

I Send Image to Face Detection Module J 



.4120 



4180 




Fig 4(a) 



Express Mail: EV330728453US 

Attorney Docket: 2100874-991440 

For: Classification System for Consumer Digital 

Images Using .... filed January 22, 2004 

Inventors: Eran Steinberg, et al., Page 8 of 28 

Attorney: Andrew V. Smith, 415-836-2522 



Core System Workflow (Workflow Module II) 




423 



4250 



[Add Faccprint to Face Class] 



424 

4280 
429' 



Add Faceprint to closest Face 
Class for this Identity 



| Link Image to Known Person Data 




j Crosslink Person to Image Data J 




4390 



Print Msg. to Message Module 



4300 




4310 



4320 
433 



4340 



Add Faceprint to most 
frequent Face Class 
for this Identity 



Back to Fig 4(a) 



Get next Faceprint in 
the Search List 



[Load Mannuai Recognition List ] 



4210 



[Get next Face on Mannuai List ) -^- 



[prompt User for Face Identity ] 



4350 




If Faceprint is valid create a new face class 

and add to Identity; Crosslink Image and 
Identity data; Pring Msg. to message module 



1355 



Update Database j 




Add Face Region 
to "Un identified" 
List 



[ Return to Main Workflow Idle j 



Fig 4(b) 
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Face Normalization Module (Mode A) 



,5010 



,5022 



Wait for Face Region from Mai 
Workflow (Idle State) 



502* 



5030 



5040 



505' 




Yes 



Determine Face 
Orientation & Pose 




Yes 



Yes 



Pass Error Message to 
Main Workflow Module 
and return to Idle 



Apply Face Feature Location Filter 



I Mark Face Features (Eye, Nose & Mouth) J 

1 \ 

\5024 



5080 



Fig 5(a) 



Apply 2-D Transforms to 
generate Frontal Face 
Region 



Estimate Stretch Factor 



Map Face Region to 3-D 
Normalized Face Model 



Rotate to Frontal Position 



Map to 2D Frontal Face 
Region 



Apply 2D Transforms for 
Illumination and Scale 



Pass Normalized Frontal 
Face Region to Workflow 
Module and return to Idle 



^5045 



-5052 

-5054 
-5056 
-5058 



-^5060 



-5070 
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Face Normalization Module (Mode B) 



Wait for Face Region from Mai 
Workflow (Idle State) 



5110 



,5122 



512 




Yes 



Determine Face 
Orientation & Pose 



5130 



5140 




Yes 



No 



5150^ 



Apply Normalization 
Filters for Scaling & 
Orientation 



Apply Face Feature Location Filter 



| Mark Face Features (Eye, Nose & Mouth) j 



^5124 



Map onto 3D Face Model 
and Generate Multi-View 
Faceprints 



-5145 



Pass Normalized Rotated 
Face Region to Workflow 
Module with Pose Rotation 
Data; return to Idle \^ 



-5160 



Fig 5(b) 
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Fig 6(d) 



Fig 6(e) 
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Fig 10(b) 
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Fig 13(a) 
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Fig 13(b) 
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Fig 13(c) 
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new ID 
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Fig 13(d) 
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